MapReduce with communication overlap (MaRCO)
نویسندگان
چکیده
منابع مشابه
MapReduce with communication overlap (MaRCO)
MapReduce is a programming model from Google for cluster-based computing in domains such as search engines, machine learning, and data mining. MapReduce provides automatic data management and fault tolerance to improve programmability of clusters. MapReduce’s execution model includes an all-map-to-all-reduce communication, called the shuffle, across the network bisection. Some MapReductions mov...
متن کاملTiling on systems with communication/computation overlap
In the framework of fully permutable loops, tiling is a compiler technique (also known as ‘loop blocking’) that has been extensively studied as a source-to-source program transformation. Little work has been devoted to the mapping and scheduling of the tiles on to physical parallel processors. We present several new results in the context of limited computational resources and assuming communic...
متن کاملMeta-MapReduce: A Technique for Reducing Communication in MapReduce Computations
MapReduce has proven to be one of the most useful paradigms in the revolution of distributed computing, where cloud services and cluster computing become the standard venue for computing. The federation of cloud and big data activities is the next challenge where MapReduce should be modified to avoid (big) data migration across remote (cloud) sites. This is exactly our scope of research, where ...
متن کاملOptimizing Metacomputing with Communication-Computation Overlap
In the framework of distributed object systems, this paper presents the concepts and an implementation of an overlapping mechanism between communication and computation. This mechanism allows to decrease the execution time of a remote method invocation with parameters of large size. Its implementation and related experiments in the C++// language running on top of Globus and Nexus are described.
متن کاملComMapReduce: An Improvement of MapReduce with Lightweight Communication Mechanisms
As a parallel programming framework, MapReduce can process scalable and parallel applications with large scale datasets. The executions of Mappers and Reducers are independent of each other. There is no communication among Mappers, neither among Reducers. When the amount of final results are much smaller than the original data, it is a waste of time processing the unpromising intermediate data....
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Parallel and Distributed Computing
سال: 2013
ISSN: 0743-7315
DOI: 10.1016/j.jpdc.2012.12.012